Robust Large Margin Deep Neural Networks

The generalization error of deep neural networks via their classification margin is studied in this paper. Our approach is based on the Jacobian matrix of a deep neural network and can be applied to networks with arbitrary nonlinearities and pooling layers, and to networks with different architectures such as feed forward networks and residual networks. Our analysis leads to the conclusion that a bounded spectral norm of the network's Jacobian matrix in the neighbourhood of the training samples is crucial for a deep neural network of arbitrary depth and width to generalize well. This is a significant improvement over the current bounds in the literature, which imply that the generalization error grows with either the width or the depth of the network. Moreover, it shows that the recently proposed batch normalization and weight normalization reparametrizations enjoy good generalization properties, and leads to a novel network regularizer based on the network's Jacobian matrix. The analysis is supported with experimental results on the MNIST, CIFAR-10, LaRED, and ImageNet datasets

Lessons from the Rademacher complexity for deep learning

Understanding the generalization properties of deep learning models is critical for successful applications, especially in the regimes where the number of training samples is limited. We study the generalization properties of deep neural networks via the empirical Rademacher complexity and show that it is easier to control the complexity of convolutional networks compared to general fully connected networks. In particular, we justify the usage of small convolutional kernels in deep networks as they lead to a better generalization error. Moreover, we propose a representation based regularization method that allows to decrease the generalization error by controlling the coherence of the representation. Experiments on the MNIST dataset support these foundations

Accurate, Very Low Computational Complexity Spike Sorting Using Unsupervised Matched Subspace Learning

This paper presents an adaptable dictionary-based feature extraction approach for spike sorting offering high accuracy and low computational complexity for implantable applications. It extracts and learns identifiable features from evolving subspaces through matched unsupervised subspace filtering. To provide compatibility with the strict constraints in implantable devices such as the chip area and power budget, the dictionary contains arrays of {-1, 0 and 1} and the algorithm need only process addition and subtraction operations. Three types of such dictionary were considered. To quantify and compare the performance of the resulting three feature extractors with existing systems, a neural signal simulator based on several different libraries was developed. For noise levels $\sigma_N$ between 0.05 and 0.3 and groups of 3 to 6 clusters, all three feature extractors provide robust high performance with average classification errors of less than 8% over five iterations, each consisting of 100 generated data segments. To our knowledge, the proposed adaptive feature extractors are the first able to classify reliably 6 clusters for implantable applications. An ASIC implementation of the best performing dictionary-based feature extractor was synthesized in a 65-nm CMOS process. It occupies an area of 0.09 mm2 and dissipates up to about 10.48 μW from a 1 V supply voltage, when operating with 8-bit resolution at 30 kHz operating frequency

Learning to Identify While Failing to Discriminate

Privacy and fairness are critical in computer vision applications, in particular when dealing with human identification. Achieving a universally secure, private, and fair systems is practically impossible as the exploitation of additional data can reveal private information in the original one. Faced with this challenge, we propose a new line of research, where the privacy is learned and used in a closed environment. The goal is to ensure that a given entity, trusted to infer certain information with our data, is blocked from inferring protected information from it. We design a system that learns to succeed on the positive task while simultaneously fail at the negative one, and illustrate this with challenging cases where the positive task (face verification) is harder than the negative one (gender classification). The framework opens the door to privacy and fairness in very important closed scenarios, ranging from private data accumulation companies to law-enforcement and hospitals

Generalization Error in Deep Learning

Deep learning models have lately shown great performance in various fields such as computer vision, speech recognition, speech translation, and natural language processing. However, alongside their state-of-the-art performance, it is still generally unclear what is the source of their generalization ability. Thus, an important question is what makes deep neural networks able to generalize well from the training set to new data. In this article, we provide an overview of the existing theory and bounds for the characterization of the generalization error of deep neural networks, combining both classical and more recent theoretical and empirical results

Vulnerability mapping as a tool to manage the environmental impacts of oil and gas extraction

Various biophysical and socio-economic impacts may be associated with unconventional oil and gas (UOG) extraction. A vulnerability map may assist governments during environmental assessments, spatial planning and the regulation of UOG extraction, as well as decision-making around UOG extraction in fragile areas. A regional interactive vulnerability map was developed for UOG extraction in South Africa. This map covers groundwater, surface water, vegetation, socio-economics and seismicity as mapping themes, based on impacts that may emanate from UOG extraction. The mapping themes were developed using a normative approach, where expert input during the identification and classification of vulnerability indicators may increase the acceptability of the resultant map. This article describes the development of the interactive vulnerability map for South Africa, where UOG extraction is not yet allowed and where regulations are still being developed to manage this activity. The importance and policy implications of using vulnerability maps for managing UOG extraction impacts in countries where UOG extraction is planned are highlighted in this article.The Water Research Commission, South Africahttp://rsos.royalsocietypublishing.orgam2018Geolog

Effects of the fatty acid amide hydrolase inhibitor URB597 on coping behavior under challenging conditions in mice

RATIONALE: Recent evidence suggests that in addition to controlling emotional behavior in general, endocannabinoid signaling is engaged in shaping behavioral responses to challenges. This important function of endocannabinoids is still poorly understood. OBJECTIVES: Here we investigated the impact of blockade of fatty acid amide hydrolase (FAAH), the degrading enzyme of anandamide on behavioral responses induced by challenges of different intensity. METHODS: Mice treated with FAAH inhibitor URB597 were either manually restrained on their backs (back test) or received foot-shocks. RESULTS: The behavior of mice showed bimodal distribution in the back test: they either predominantly showed escape attempts or equally distributed time between passivity and escape. URB597 increased escapes in animals with low escape scores. No effects were noticed in mice showing high escape scores, which is likely due to a ceiling effect. We hypothesized that stronger stressors would wash out individual differences in coping; therefore, we exposed mice to foot-shocks that decreased locomotion and increased freezing in all mice. URB597 ameliorated both responses. The re-exposure of mice to the shock cage 14 days later without delivering shocks or treatment was followed by reduced and fragmented sleep as shown by electrophysiological recordings. Surprisingly, sleep was more disturbed after the reminder than after shocks in rats receiving vehicle before foot-shocks. These reminder-induced disturbances were abolished by URB597 administered before shocks. CONCLUSIONS: These findings suggest that FAAH blockade has an important role in the selection of behavioral responses under challenging conditions and-judging from its long-term effects-that it influences the cognitive appraisal of the challenge

The clinical utility of molecular diagnostic testing for primary immune deficiency disorders: a case based review

Primary immune deficiency disorders (PIDs) are a group of diseases associated with a genetic predisposition to recurrent infections, malignancy, autoimmunity and allergy. The molecular basis of many of these disorders has been identified in the last two decades. Most are inherited as single gene defects. Identifying the underlying genetic defect plays a critical role in patient management including diagnosis, family studies, prognostic information, prenatal diagnosis and is useful in defining new diseases. In this review we outline the clinical utility of molecular testing for these disorders using clinical cases referred to Auckland Hospital. It is written from the perspective of a laboratory offering a wide range of tests for a small developed country

Modulation of the endocannabinoids N-Arachidonoylethanolamine (AEA) and 2-Arachidonoylglycerol (2-AG) on Executive Functions in Humans

Animal studies point to an implication of the endocannabinoid system on executive functions. In humans, several studies have suggested an association between acute or chronic use of exogenous cannabinoids (Δ9-tetrahydrocannabinol) and executive impairments. However, to date, no published reports establish the relationship between endocannabinoids, as biomarkers of the cannabinoid neurotransmission system, and executive functioning in humans. The aim of the present study was to explore the association between circulating levels of plasma endocannabinoids N-arachidonoylethanolamine (AEA) and 2-Arachidonoylglycerol (2-AG) and executive functions (decision making, response inhibition and cognitive flexibility) in healthy subjects. One hundred and fifty seven subjects were included and assessed with the Wisconsin Card Sorting Test; Stroop Color and Word Test; and Iowa Gambling Task. All participants were female, aged between 18 and 60 years and spoke Spanish as their first language. Results showed a negative correlation between 2-AG and cognitive flexibility performance (r = −.37; p<.05). A positive correlation was found between AEA concentrations and both cognitive flexibility (r = .59; p<.05) and decision making performance (r = .23; P<.05). There was no significant correlation between either 2-AG (r = −.17) or AEA (r = −.08) concentrations and inhibition response. These results show, in humans, a relevant modulation of the endocannabinoid system on prefrontal-dependent cognitive functioning. The present study might have significant implications for the underlying executive alterations described in some psychiatric disorders currently associated with endocannabinoids deregulation (namely drug abuse/dependence, depression, obesity and eating disorders). Understanding the neurobiology of their dysexecutive profile might certainly contribute to the development of new treatments and pharmacological approaches